Gaze-Contingent Spatio-temporal Filtering in a Head-Mounted Display

The spatio-temporal characteristics of the human visual system vary widely across the visual field. Recently, we have developed a display capable of simulating arbitrary visual fields on high-resolution natural videos in real time by means of a gaze-contingent spatio-temporal filtering [1]. While such a system can also be a useful tool for psychophysical research, our main motivation is to develop gaze-guidance techniques. Because the message an image sequence conveys depends on the exact pattern of eye movements an observer makes, we propose that in future information and communication systems, images will be augmented with a recommendation of where to look, of how to view them. Ultimately, we want to incorporate gaze guidance technology into mobile applications; such technology, integrated into a head-mounted display (HMD), could use computer vision techniques to enhance human visual performance. In our demonstration, we will show a first implementation of such a device in the form of a system that implements our gaze-contingent spatio-temporal filtering algorithm in an HMD with video-see-through. Subjects will be able to walk around, seeing their natural visual environment inside the HMD. We will demonstrate that we then can manipulate what the subjects see in real time.